10 / 10

What versioning strategy would you use so that switching to a new embedding model doesn't silently corrupt search quality for existing data?

Dual-write to a new collection or named vector, cut over only when fully re-embedded

The strategy is to treat the embedding model as a versioned property of the collection, not as a runtime detail. When you switch models, you create a new collection (or a new named vector within the same collection) with the new model's dimension and metric, and you dual-write new content to both the old and the new collections during the migration. You backfill the old collection's content into the new collection by re-embedding it with the new model, and you track progress so you know when the backfill is complete. Only when the new collection is fully populated and validated do you cut over the query path to it, typically by moving an alias. The old collection remains available as a rollback target until the new one has been stable for a defined period. This avoids the two failure modes of an in-place migration: mixing old and new vectors in the same collection, which produces a mix of representations that no query can rank correctly, and cutting over before the backfill is complete, which silently returns incomplete results.

The mechanism that makes this safe is that the old and new collections are separate vector spaces, and the query path always uses exactly one of them. During the migration, new writes go to both, and old data is backfilled into the new collection. If the backfill is slow, the old collection continues to serve queries correctly (it has all the data, just in the old space), and the new collection is incomplete but not yet used for queries. The migration state is therefore always consistent from the query path's perspective: it either uses the old collection (complete, old space) or the new collection (complete, new space), never a mix. The cutover is a metadata operation (moving an alias) and can be rolled back by moving the alias back. The validation before cutover compares recall on the new collection against a held-out query set, and compares the new collection's results with the old collection's results on a sample of queries to ensure that quality has not regressed. This is the standard pattern for any schema or model change that cannot be done in place.

  1. 1

    New collection per model version: the new model's dimension and metric are set at creation.

  2. 2

    Dual-write during migration: new content is written to both collections.

  3. 3

    Backfill: re-embed existing content with the new model and upsert into the new collection.

  4. 4

    Progress tracking: know when the backfill is complete before cutting over.

  5. 5

    Validation: compare recall on the new collection against a held-out set, and compare rankings with the old collection.

  6. 6

    Cutover via alias: a metadata operation that can be rolled back.

  7. 7

    Rollback window: keep the old collection available until the new one is proven stable.

  8. 8

    Named vector alternative: within a single collection, add a new named vector field for the new model and migrate to it.

The trade-off is between the cost of running two collections during the migration and the safety of never mixing vector spaces. The dual-write and backfill require extra storage and compute, and the migration takes time proportional to the corpus size and the embedding throughput. The alternative - an in-place migration where you update the collection's config and re-embed - is not possible in Qdrant because the dimension and metric are immutable, but even if it were, it would be unsafe because during the migration the collection would contain a mix of old and new vectors. The common mistake is to try to run both models against the same collection by storing the model version in the payload and filtering, which does not work because the vector spaces are incompatible. The second mistake is to cut over before the backfill is complete, which silently returns incomplete results. The third mistake is to skip the validation step, so a model that is worse than the old one is deployed. The fourth mistake is to delete the old collection immediately after cutover, leaving no rollback path. Version note: the alias API and the collection creation API have been stable, but the exact shape of the alias operations and the supported named vector configurations have evolved. The named-vector alternative (adding a new vector field to an existing collection) is a newer capability and its API may differ across versions.

javascript

Version-dependent: the alias API and the collection creation API have been stable, but the exact shape of the alias operations and the support for named vectors with different dimensions have evolved. In some versions, a collection can have multiple named vectors with different dimensions and metrics, which enables an in-collection migration by adding a new named vector and gradually moving queries and writes to it. The query_points API is qdrant-client 1.10+; older clients used search(). Verify the migration capabilities on your version before planning.

Difficulty: 8/10
Topics: Embedding Models, Migration, Versioning

Scenario Questions

0-2 years experience
  1. 1

    You switch embedding models and query results become inconsistent. Explain the cause and how versioning would have prevented it.

  2. 2

    A teammate tries to run both models against the same collection by filtering on model_version. Explain why this does not work.

2-5 years experience
  1. 1

    You are migrating a 10M-point collection to a new model. Describe the dual-write, backfill, validation, and cutover steps, and how you handle the window where the new collection is incomplete.

  2. 2

    You cut over to the new collection and discover a quality regression. Describe your rollback plan and how you would prevent the same issue next time.

5-8 years experience
  1. 1

    Design a migration pipeline that keeps the old and new collections in sync during a multi-day backfill, including how you handle updates and deletes that occur during the migration.

  2. 2

    You need to migrate with zero downtime and a 1-hour rollback window. Describe the architecture, the automation, and the validation.

8+ years experience
  1. 1

    You are designing a system where the embedding model is upgraded every quarter and the corpus is 500M points. Describe the migration architecture, the automation, and how you validate quality before each cutover.

  2. 2

    Derive the cost of a model migration as a function of corpus size, embedding throughput, and the required redundancy. How would you decide between a full parallel deployment and an in-collection named-vector migration?

Follow-up Questions

  • How would you track and validate the progress of the backfill, so that you know when it is safe to cut over?
  • What would you do if the new model is worse than the old one on some queries but better on others? How would you decide whether to cut over?